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Abstract 

We study here the enumeration problem of permutations which sat- 
isfy certain additional constraints. Given a class of permutations K a 
pattern /i and a fixed integer j, we ask for the number of permutations 
avoiding /i whose biggest sub-permutation in K, has size bounded by j. 
We provide several new results considering different instances of this 
problem depending on n and /C. In particular, we derive enumerations 
when the avoided pattern is 312, 123 and 1-32 and when the con- 
sidered test sets JC are also of pattern avoidance type. Most (but not 
all) of the cases studied correspond to interesting sub-tree properties of 
binary trees. In this sense, by the use of pattern avoiding conditions, 
we extend a problem considered by Flajolet et al. in a previous work. 

1 Introduction 

The aim of this paper is to provide several enumerative results study- 
ing sub-permutations. Sub-permutations, as defined in Section [2J cor- 
respond to the classical concept of sub-tree according to a well known 
bijection which maps the set of pemutations of size n onto the set of 
binary increasing trees having n nodes. 

The main section of the paper, i.e. Section [3l is dedicated to solve 
some instances of the following general problem. 

Problem: given a class of permutations /C, what is the number of 
permutations it whose biggest sub-permutation in K, has size bounded 
by a fixed number? 

If the permutation tt does not have to satisfy any pattern avoiding 
condition the problem has been addressed by Flajolet, Gourdon and 
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Martinez j5]. Here, we introduce the condition of pattern avoidance 
and derive several novel results when the avoided pattern (fj.) is of 
length three. More precisely, we consider = 312, ^2 = 123 and 
/i3 = 1 — 32 where the first two are classical Catalan patterns and the 
third is a generalized pattern according to [T]. 

In Section [3TT] we consider four possible instances of class /C. For 
III we consider JC = Av{213) and IC equal to the set of odd alternating 
permutations. In particular we prove that, for n — > 00, taking at 
random a permutation in tt S Aw„(/ii) the expected value of the size 
of the biggest sub-permutation of n belonging to ^■i;(213) goes hke 
log2(n). For /i2 we consider IC = Av{21) and K, = Av(12) showing 
how, in the last case, the number of permutations of ^w„(/i2) whose 
biggest sub-permutation in IC has size bounded by a fixed number j 
is related to the number of Dyck paths of size n avoiding the pattern 

In Section [3. 21 we focus on permutations avoiding fi^ characterizing 
them in terms of paths of the associated trees. We then consider IC = 
Av{l2) and provide a limit approach to the enumeration of AvnifJ-s) 
which is given by Bell numbers. 

In Section[3]we introduce the concept of pattern avoidance in terms 
of sub-permutations. We find the probability to detect a pattern 213 
in a permutation looking just at the sub-permutation generated by 
the entry 2 and we generalize our result considering any pattern a. 
We think this new kind of pattern related problems deserves further 
studies. Indeed it seems intersting to know in which cases one can 
predict properties of a permutation when it is allowed to inspect only 
a limited part of it. 

Finally, we note that the kind of questions formulated here in the 
context of permutations are generic for all combinatorial objects for 
which a notion of sub-structure is defined. 

2 Preliminaries 

The set of permutations of size n is denoted by iS„ and S = Un*^"- 
We assume the reader to be familiar with the classical concept of pat- 
tern avoidance in permutations. The subset of iS„ made of those per- 
mutations avoiding the pattern a is usually denoted by Avn{cr) and 
M'y) = U„(^^n(a)). 

Let TT — (7ri7r2 . . . 7r„) be a permutation. For a given entry tt^ 
we define ST(7ri) as the biggest sub-string of tt which contains tt^ 
and whose entries are greater than or equal to tt^. Furthermore let 
gTi{TTi) be the permutation obtained rescaling s,r(7ri). We call g-wi'Ki) 
the sub-permutation of tt generated by tt^. The set of sub-permutations 

Sub-strings are made of adjacent elements of tt. 
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Figure 1: (a) The permutation vr = (45312687); (b) the tree associated with 
TT by (j). 

(57r(7''i))i=i...n is denoted by Gtt- As an example consider the permu- 
tation TT which is depicted in Fig. [T] (a). In this case is made 
of 5^(5) = <?,(8) = (1), ff^(4) = (12), 3,(7) = (21), g„(3) = (231), 
5,(6) = (132), 3,(2) = (1243) and ^,(1) = (45312687). 

Given a class of permutations K, we denote by 77r(/C) the size of the 
biggest sub-permutation of tt belonging to /C. 

The concept of sub-permutations is related to the one of sub-trees. 
To illustrate this correspondance we make use of a well known bijection 
between the set Sn and the set of binary increasing trees of size n, 
denoted by Tn- We recall that a planar rooted tree i, having n nodes, 
belongs to Tn when: 

- each node has outdegree 0, 1 or 2. Nodes of outdegree are called 
leaves; 

- each node (except for the root) can be left or right oriented with 
respect to its direct ancestor (see Fig.[T](b)); 

- each node is labelled (bijectively) with a number in {1, in 
such a way going from the root of t to any leaf of t we find an 
increasing sequence of numbers. 

The bijection (j) : Tn ^ Sn is given by the following procedure: 

i) given a tree t each leaf of t collapses into its direct ancestor whose 
label is then modified receiving on the left the label of the left 
child (if any) and on the right the label of the right child (if any) . 
We obtain in this way a new tree whose nodes are labelled with 
sequences of numbers; 

a) starting from the obtained tree go to step i). 

The algorithm (j) ends when the tree t is reduced to a single node 
whose label is then a permutation (j3{t) of size n. For an example see 
Fig. [II 
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The link between sub-permutations and sub-trees is expressed by 
the following proposition. 



Proposition 1 Given a permutation tt ~ (t>{t), let ti he the (re-scaled) 
sub-tree oft generated by the node n^, then the sub-permutation gTrij^i) 
is equal to 4>{ti). 



3 On the biggest sub-permutation of tt G 
Av{^) belonging to /C 



In this section we want to determine the number of permutations in 
■^Vn (m) with the condition that the size of the biggest sub-permutation 
in K. is bounded by a fixed number. We consider several instances of 
and K, and find new enumcrative results. Some of them are connected 
to already known combinatorial problems. 

3.1 Two classical patterns of length three 

In this section we study the patterns /i = 312 and = 123. It is well 
known that both the classes ^'i;(312) and Av{122>) are enumerated by 
Catalan numbers whose first terms are 

Co = 1, ci = 1, C2 = 2, C3 = 5, C4 = 14, C5 = 42, cg = 132, C7 = 429. 

The associated ordinary generating function is ^6] 



if n is large. 

3.1.1 /U = 312 and binary rooted planar trees 

We denote by Bn the set of binary rooted planar trees with n internal 
nodes, where each internal node has outdegree two. It is well known 
that one can bijectively map the set Bn onto the set Aw„(312). In 




(1) 



which has a closed form for its n-th coefficient given by 




Furthermore, it is well known that 
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Figure 2: The mapping tl^. 



particular we use a bijection ip : Bn — >■ Aw„(312) which works similarly 
to the mapping of Section [5J as follows. 

Take t & Bn and visit its nodes according to the pre-order traversal 
labelling each node of outdegree two in increasing order starting with 
the label 1 for the root. After this first step one has a tree labelled 
with integers at its nodes of outdegree two. Each leaf now collapses to 
its direct ancestor which takes a new label receiving on the left (resp. 
right) the label of its left (resp. right) child. We go on collapsing 
leaves until we achieve a tree made of one node which is labelled with 
a permutation of size n. See Fig. [5] for an instance of this mapping. 

K, — Av{213) and caterpillar sub-trees 

Let now JC — Av{213). Through the bijection tp we see that, given 
a permutation tt e Av(3l2), the parameter 77r(/C) corresponds then to 
the size (number of internal nodes) of the biggest sub-tree of ip~^{Tr) 
which is a caterpillar: a tree t in Bn is a caterpillar of size n, if each 
node of Hs a leaf or has at least one leaf as a direct descendant. 

If we define Pj as the ordinary generating function 



counting those permutations tt in Avn{^J,) having 77r(^) < j, we observe 
that Pj must then satisfy the equation 



Indeed, a tree t whose biggest caterpillar sub-tree is of size at most 
j, is either a leaf or it is built by appending to the root of t two trees 
ti and t2 of the same class. We must exclude the case in which one 
of the two, ti or t2, has size 0, i.e., is a leaf, and the other one is a 
caterpillar of size j. Since there are exactly 2^~^ caterpillars of size j 
the previous formula follows. 

From ^ we obtain 



V 



Pj = l + xP.^ - Vx' 



(2) 




1 - Vl - 4a; + 2J+2a;J+2 



(3) 
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which gives an expansion, for j — 1, with the foUowing coefheients 

1, 1, 0, 1, 2, 6, 16, 45, 126, 358, 1024 

which correspond to sequence ^1025266 of [5]. As stated there, they 
also count classes of Motzkin paths satisfying some constraints. 

The asymptotic behaviour of the coefficients of Pj follows by stan- 
dard analytic methods [6j . When n becomes large the coefficients grow 
like 



1 4p,-(j + 2)2^+2pf2 / I 



n+l 



(4) 



4 V T^n-^ \Pj 

where pj is the smallest positive solution of 

1 - 42; + 2^+2x^+2 = 0. 

Starting from the following inequality 

1 2 

4<^^<5' 

which is strightforward to prove for all j > 1, and applying the so- 
called bootstrapping technique [B] one can show that pj tends to 1/4 
as 
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The main result we want to present in this section concerns the 
average of 7 = 77r(^) when the size of the permutation tt is large. 

Proposition 2 When tt £ AvnifJ-), expected value of ^^{K,) is 
asymptotically equivalent to log2(n). 

Proof. If n > 1 we can express the desired average value as 

lwi,n + Ej>l(i + l)(^'i-H 



End) 



-Vl.n - ■■■ - Vn-l,n + nv^.^ + J2j>nij + ^){Vj+l,n " Vj^n) 
-Viji - ... - Vn-l,n + nCn + I]j>„(Cn - Wj,„) 



1 H — 
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In the previous calculation we have used the fact that for j > n we 
always have Wj,„ — c„. 

It is sufficient now to find the ?i-th term of the function 



In what follows we want to find a function U which estimates U 
near the dominant singularity 1/4. According to 6 , when n is large, 
the n-th term of the Taylor expansion of U provides an approximation 
of [x^]U{x). Our approach results to be similar to the one used in 
Section 3 of [7]. 

Let us fix X near 1/4 and let us consider the threshold function 




jo = logs 



1 



ii-4xr 



Then, supposing j > jo ~ 1: we have that 




while if j < jo — \ wc can use the approximation 




For X sufficiently close to 1 /4 we estimate U {x) as 
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Uix) 



JO -2 

E 



2a;Vl - 4a; ^ V 2^+2 



1 VI -4a: 



2x 



JO -2 

El 

j>i 



+ 



yi -4x 

2.t(1 - 4x) ^ 2J+3 
JO -2 



E - 

/-^ 9J+ 



-1 J I' 

-E 

At 



Ax ^ \ 2^ 

J>1 



1 Vl - 4a; 



2x 



(io - 2) + 



E - 

^ 9J 



1 -\/2 + 2V2\/2^ 



4a; 



-2 + V2 



Vl-4x 
2x 



logs 



1 -4a; 



- 2 



16a;v'l - 4a; ./^ , 2^ 

J>Jo-l 



22-io 



16x^1 - 4a; 



log(2) %/21og(2) 2V2-8a;log(2) 



a;log(16) a;log(16) a;log(16) 

7T^4ilog(2) 2V1 -4a;log(^^) 
a;log(16) a;log(16) 
271^ log (^) 
a;log(16) 



Using the previous calculation we can say that 



2yr^iog(^ 

^ .log(16) ' 

approximates [/(a;) near its dominant singularity 1/4. It follows that 
when n ^ oo 

Applying standard methods O [7] to ([5]) we find that 
[a;"]f/(a;) - 



log(16)v7m'^ 

Dividing by the asymptotic behaviour of Catalan numbers gives the 
claim. □ 

As a test one can consider the following table where, for several 
values of n, we compare the true -En (7) with the approximation given 
by Proposition [21 
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n 


10 


20 


50 


100 


200 


500 


1000 


End) 
l0g2(?l) 


3.596 
3.321 


4.172 
4.321 


5.227 
5.643 


6.121 
6.643 


7.058 
7.643 


8.336 
8.965 


9.319 
9.965 



To conclude this section it is interesting to observe another possible 
apphcation of the function Pj described in ([3|). Indeed the number of 
permutations in AvnifJ-) having no sub-permutation of size j in /C is 
given by the n-th coefficient of 



= ^ . (6) 



1 - Vl-4a; + 2J+i.TJ- 
"2x~ 

We will compare this result with the analogous one provided in the 
next section. 

When K, is the set of (odd) alternating permutations 

A tree in Bn, where n is odd, is called strictly binary if, removing 
the leaves, the remaining nodes have either out-degree or out-degree 
2. The corresponding (through the mapping ip) sub-set of AvnifJ-) 
consists of permutations vr ~ (7ri7r2...7r„) characterized by the following 
property: either n = 0, 1 or vri > 7r2 < tts > ... < 7r„ . It is well known 
that the number of strictly binary trees of size 2m + 1 is Cm . 

The parameter jTr{lC) corresponds in this case to the size of the 
biggest strictly binary sub-tree of i/;~^(7r) and, obviously, the only pos- 
sible values of 'Jtt{IC) are odd. The ordinary generating function Lj{x), 
with j — 2m + 1, counts those trees in Bn having at least one strictly 
binary sub-tree of size j. Equivalently, it can be seen as the function 
counting the permutations tt in AvnifJ.) with 77r(/C) > j- Lj must then 
satisfy 

Lj = CmX^ + a;L/ + 2xLj{C - Lj), (7) 

where C is the generating function of Catalan numbers as in ([1]). 

Indeed a tree counted by Lj is either a strictly binary tree of size 
j (first summand) or it is built by appending to the root two trees, 
where at least one of them contains a strictly binary sub-tree of size j 
(second and third summand). 

Solving ([7]) we obtain 



L,{x) = — . (8) 

From this, we can also determine the number of those permuta- 
tions avoiding the pattern 312 and without (odd) alternating sub- 
permutation of size j . This is given by 



C{x) - Lj {x) = — . (9) 



9 



We now want to compare results ^ and ([9]) for a fixed j — 2m + 1 
with m large. Let us consider > 1/4 (resp. 5,„ > 1/4) defined as 
the smallest positive root of 1 — 4a: + icmx'^"^'^'^ — (resp. 1 — 4a; + 
22m+2^2m+2 _ gy asymptotic considerations, we know that when 
m — >■ CXI both am and 6„i tend to 1/4. But we can be more precise: 
when TO is large, the equality 

1 - 4a„ + 4c„,a^™+2 = 0=1- 46„ + 2^^+%^^^^ 

implies 

a^^-i+4™6|r^ -i + {ir+' 

brn -1+C„a^"'+1 ^ -1 + C,„(i)2™+1 

For example, if m = 5, we have that am/bm ~ 0.999765 while the 
previous approximation gives 0.999766. 

By standard methods (see [6]) one can compute the asymptotic 
behaviour of the n-th coefficient of (jH]) as 



1 / 4a^- (2m + 2)40^0^"+^ 

while the behaviour of the n-th coefhcient of ([6]) is given by (|4]) (con- 
sidering bm = Pj-i) as 



1 Ubm- (2to + 2)22™+26^™+2 / 1 



n+1 

4 y Trn'^ V bn 

From these results follows 

Proposition 3 For a fixed and sufficiently large m, when n — >■ oo, the 
ratio 

|{7r e AvnifJ,) ■■ (G^ n ^i;(213) n 52™+i) 



|{7r € Avnin) : (G^ n /C n 52m+i) = 
^oes to equally fast as 



(10) 



where km is a constant depending only on to. 

In the following table we compare the values of the ratio of Propo- 
sition [3] with the asymptotic ratio (fTTj) for to = 5 and different values 
of n. 



n 


50 500 1000 5000 10000 


ratio (fTOj) 
A:™ = 1 in (dH 


0.986 0.887 0.789 0.308 0.095 
0.988 0.889 0.791 0.310 0.096 
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Figure 3: The two-line representation of (78436521) G ^^§(123); for example 
6 covers 4 and 3 

3.1.2 /i = 123, a generating tree approach 

In what follows let /i ~ 123 be fixed. It is a well known fact that 
the entries of a permutation vr G can be seen as points lying 

on two non-intersecting lines as depicted in Fig. [3l Viceversa, each 
permutation which can be drawn in such a way avoids /i. A point pi 
on the line D is covered by a point p2 of the upper line U if p2 is on 
the right and above pi . In order to avoid redundancies in the two-line 
representation of a permutation we have to respect the following rule: 
a point belongs to the upper line U if and only if it covers at least one 
point of D. 

It is useful to observe that the set Aw„+i(/i) can be generated by 
the permutations of ylw„(/i) adding the rightmost entry. Taken tt g 
AvnifJ'), US define u(7r) as the right-most point placed on the line U 
(if any) and let 1{t:) be the number of elements which are placed on D 
with a smaller ordinate than u{t:). In order to create a permutation tt' 
of size n + 1 we add on the right of tt a new element p' . If p' is placed 
on the D line then 1{tt') = l{Tr) + 1. Otherwise p' can be placed on 
the U line in exactly 1{tt) different ways, see Fig. 01 In this case 1{tt') 
ranges between 1 and Z(7r). 

In conclusion, we have described a generating tree procedure which 
starts with the single element permutation and, always adding the 
rightmost entry, generates all the permutations in Av{123). Further- 
more, if we consider (/) = (Kti")) the previous procedure can be sum- 
marized as 
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(2) 



(2) 



(1) 



(3) 



Figure 4: The permutations produced starting from (45231). The new right- 
most entry is highUghted. 



This rule can be viewed as a tree generating procedure, where a 
label (l) is a node with (/ + 1) descendants. If the starting label (i.e. 
the root of the tree) is (0), the number of nodes at level n is given by 
c„. More details on generating trees can be found in [2]. 

K, — Av{21), the biggest increasing sub-permutation 

In this section 77r(/C) G {1, 2} is the biggest increasing sub-permutation 
of a given tt € Av{ii) and we notice that Jtt{I^) = 2 if and only if the 
first (from left to right) three entries of tt are in the order 231. Indeed, 
if a < 6 are the entries of an increasing sub-permutation of size 2, such 
a sub-permutation is the one generated by a. It follows that a must 
be the leftmost entry of tt while b the topmost. In order to 'close' the 
sub-permutation, we also need the presence of a point c < a placed 
immediately on the right of b. 

To count the permutations tt G Av{fj.) of size n(7r) > 3 with the 
biggest increasing sub-permutation of size 2 we have just to consider 
rule (fT2| with a starting point given by (2) = ^(231). 

By standard computations, see again [2l , one finds that the ordinary 
generating function 



(0-(l),(2),...,(?-t-l). 



(12) 



M2 = M2(x,2;) = ^:e"Wj/'W 



TT 



satisfies the functional equation 




(13) 



which can be solved by the kernel method finding 
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A closed formula for the coefficients is given by 

[x-]M{x, 1) ^ ^^^^^^ (withn > 3). 
(n — 6)\n\ 

In conclusion we can state that 

Proposition 4, If 7i > 3, the number of permutations in Au„(123) 
having an increasing sub-permutation of size 2 is 

_ 3(2n-4))! 
" (n — 3)!n! 

Furthermore, the ones having the biggest increasing sub-permutation 
of size 1 are ( with n > 3) 

where Cn is the n-th Catalan number. 

For n — 3, 10 the following table shows the values of a„ and 6„. 



n = 


3 


4 


5 


6 


7 


8 


9 


10 


an 


1 


3 


9 


28 


90 


297 


1001 


3432 


bn 


4 


11 


33 


104 


339 


1133 


3861 


13364 



The coefficients a„ are the (shifted) entries of sequence A000245 of 
[9], while the numbers bn do not appear there. 

IC = Av{12) and pattern avoidance in Dyck paths 

In this section "f-nilC) is the biggest decreasing sub-permutation of 
a given tt G Av{ij,). Looking at the two line representation of tt, which 
is described in Section 13.1.21 we see that 5^(7ri) is decreasing if and 
only if it corresponds to a sequence of adjacent points (adjacent with 
respect to their abscissas) lying on the line U, with the only exception 
of TT € Av{12). For example, the permutation depicted in Fig. |3]has 
three decreasing sub-permutations, the one generated by the entry 8 
the one generated by 6 and the biggest one which is generated by 5. 

The relation between 7ii-(/C) and the size of the biggest sequence of 
adjacent points on the line U, denoted by 7^, is given by 

7.(/C) -7^, if ^ ^ Av{12) 
7^(/C) =n, if TT e Avn{l2) 
7^ =0, if TT e yli;(12) 

It follows that we can easily relate the number of permutations having 
7^(/C) = j with the number of permutations with 7^ — j. The number 
of tt's such that 7^ is at most j is the subject of study in this section. 
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Given tt, it is useful to consider the point d(7r) defined as the right- 
most point placed on the line D. With the terminology of Section [3.1.2[ 
the parameter v — vijr) is defined as the number of points placed on 
the line U which are covering diir). In the permutation of Fig. [3] we 
have w(7r) — 0. Looking at the recursive contruction of Av{fi), which 
has been defined in Section [3.1. 2[ we observe that, in order to create 
only permutations satisfying < J, it is sufhcient to avoid at each 
step the construction of a permutation with a w-value greater than j. 
This corresponds to use the rules of the form 

(/) {I') (with /' < /) (14) 

no more than j times consecutively. 

If we put the same restrictions on a particular recursive construc- 
tion holding for Dyck paths, it turns out that the statistic 7^ < j is 
equivalent to determine the number of those paths avoiding the pat- 
tern U^~^^D (defined below) and having a fixed size. This problem on 
paths has been deeply studied in the literature, see for example [5] as 
well as the following sequences of [9]: A001006, i.e., the one of Motzkin 
numbers, for the case j = 1, ^036765 for the case j = 2 and ^036766 
for j = 3. 

We assume the reader is familiar with the definition of Dyck paths. 
The set of Dyck paths of semi-length n is denoted by Dn and it is a 
well known fact that |I?„| = c„. We represent a path p as a sequence of 
up U and down D steps. We define a block of p as a minimal sub-string 
(made of consecutive entries) which is still Dyck path and which starts 
at height in p. For example, if p = UUDUDDUD, then there are 
only two blocks inside p. More precisely, we find UUDU DD which is 
made by the first six steps and UD which corresponds to the last two 
entries of p. 

The set X'n+i can be generated recursively by the paths in I?„ 
according to a construction which we use to show the main result of 
this section. Let 

p = bi...bk-ibk 

be a path of I?„ decomposed in terms of its blocks, where fc > 0. In 
order to create a path p' of semi-length n + 1 we add two steps U and 
D inside p in /c -|- 1 possible ways. Obtaining 



p' ^UDp or (15) 
p' ^Ubi..MDb,+i...bk {l<i<k). (16) 

In both cases we add an U step at the beginning creating the left- 
most block of p' . If p is a path, let l{p) be the number of its blocks. 
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The construction above corresponds then to the generating tree asso- 
ciated with the rules already described in (IT^ . In particular, P3|) is 
associated with (1) {I + 1). 

Now, we are ready to prove the following result. 

Proposition 5 The number of permutations tt G Avn^i) having 7^ < 
J equals the number of paths in 2?„ avoiding U^^^D 

Proof. Considering what we have already shown previously, we have to 
prove two facts: i) if we use the rule (l) {I') with /' < I for more than 
j consecutive times in the construction of a path p, then p contains the 
considered pattern; ii) if p contains U^^'^D, then it has been created 
using rule p4)) consecutively at least j + 1 times. To prove i) it is 
enough to observe that we obtain a path which starts as U^^^bi... if 
we start from a generic path bi...bk and we apply fH)) j + 1 times 
consecutively. The claim follows because bi starts as 61 = WD..., with 
i > 0. To prove ii) let us suppose that U^~^^D is inside a path p 
and let us denote by T the generating tree associated with rules (IT5|) 
and (jl6p . Observe that we can move inside T going from p to the 
root simply removing, step- by-step, the left-most entry U oip and the 
corresponding D. We must find, at some point, an ancestor of p which 
starts as U^^^D.... To buid this ancestor the last j + 1 applications of 
the construction rule belong to case (fT6|) . □ 

3.2 = 1 — 32 and right-left avoiding trees 

In this section we consider the permutations avoiding the generalized 
patterns 1 — 32. According to T a permutation tt contains the pattern 
1 — 32 when there are three indices ji < j2 < js such that = j2 + 1 
and upon rescaling TTjjTTjjTTjg one obtains the pattern 132. 

The mapping (j) described in Section [2] gives a nice representation 
of these permutations in terms of the associated trees. A tree i S 7^ is 
said to be a right-left avoiding if all nodes of t can be reached starting 
from the root of t with a sequence of steps which does not contain a 
left step after a right one. We denote by C the set of right-left avoiding 
trees. An example of a tree in £10 is depicted in Fig. [S] (a) where the 
associated permutation is 

TT = (10856947213). 

Observe that from tt we can extract the pattern 132 from the entries 
6, 9, and 7 but 9 and 7 are not adjacent in tt. Indeed, we can prove 

Proposition 6 A tree t Cz T is right-left avoiding if and only if (j){t) G 
Av{l - 32). 
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10 

(a) 



(b) 



Figure 5: (a) A tree in £10, right oriented chain sub-trees are highlighted; 
(b) all the possible situations which can produce a right (from a to 6)-left 
(from b to c) sequence of steps in a tree. 

Proof. We start showing that if 0(i) € Av{l — 32), then t is right-left 
avoiding. In Fig. [5] (b) we have sketched all the possible situations giv- 
ing a tree which is not right-left avoiding. Indeed, in order to reach c 
from a, we have to perform a right-left change of direction. The corre- 
sponding pattern in the associated permutation is then vuayczbxw 
(where v and w cannot appear at the same time). Observe that we 
find the entries a, b and c giving a pattern 1 — 32 except when the entry 
z exists. But, in this case, we can always find a node of the sub-tree 
generated by z which gives, together with a, b, a pattern 1 — 32. 

Vice versa, let us take t in C and let us suppose that there are three 
entries of </'(t), i.e., T^ji,'i^j2 and TTja+i with ji < j2 giving the pattern 
1 — 32. Observe then that the most recent common ancestor of tt^-j and 
TTjj+i in the tree t must be Tr^a+i. Furthermore, looking at the left and 
right orientation of the nodes of i, we see that TVj^ belongs to the left 
sub-tree of TTja+i. Thus, we must perform a left step to reach -Kj^ from 
77^2+1. This implies that the entire path from the root of t to tTj^+i is 
made only of left steps. Thus, there is no place for the entry Hj^ . □ 

JC = Av{\2), towards Bell numbers 

Let TT € Avn{lj)- In this section the parameter 7,r(A^) corresponds to 
the size of the biggest chain sub-tree of (j)~^{-K) which is right oriented. 
Right oriented chain sub-trees are highlighted in Fig. [5] (a) and, if tt is 
the permutation associated with the right-left avoiding tree depicted 
there, then 77r(/C) = 2. In what follows we fix j > 1 and we determine 
the value of Ij^n defined as the number of trees in having no right 
chain sub-tree bigger than j. 

In order to determine the numbers Ij^n one can proceed using a 
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recursive construction. A tree t in Cj can be decomposed according to 
the following cases 

i) t is the empty tree; 
ii) i is a right chain tree and 2 < |t| < j; 

Hi) t is obtained appending two trees to a common root. On the 
right we attach a right chain tree t' with 1 < < j while on 
the left we put a non-empty tree of Cj ; 

iv) i is a root to which we append on the left a tree of Cj . 

From the previous decomposition follows that the exponential gen- 
erating function 

n>0 

satisfies 



L, =1+ (y + - + ^) + j ^i'^^ 



in + 2)\\ 1 / ■■' ^{n+i + l)\\ j 

n>0 ^ ' ^ \ / n>0 ^ / \ -J 



Hi) 



from which we obtain the derivative 



L; =L,[l + x + - + ... + -y -, L,(0) = 1. (17) 
Solving PT)) we find that 



L,{x) ^ E,{x) + E,{x) I -^L^^dz-E,{x) I -r^^dz, (18) 



where 



Now recall that we are interested in the coefficients 



and observe that 
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j\E,{z) 



dz 



in) 



(x) ^Ef\x) 

n-1 

E 



X 



n-l 



■dz 



i=0 L 



(x). 



Then, from p^. we have 



J'- 



i=0 




(n-l-i) 



(x) 



E,{x) 



(0). (19) 



The following table shows the values of Ij^n for j = 1,2,3 and 
1< n < 10. 



n 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


j 


= 1 


1 


1 


3 


6 


18 


48 


156 


492 


1740 


6168 


3 


= 2 


1 


2 


4 


13 


41 


146 


587 


2470 


11254 


54616 


j 


= 3 


1 


2 


5 


14 


50 


190 


800 


3670 


18190 


95980 



Formula ([19)) highlights the connection which holds between the 
sequence (of sequences) {lj,n)j and a family of interesting generating 
functions, namely {Ej)j. The function Ej appears in the literature 
several times. For example, in [4], it is related to the problem of 
counting the number of permutations belonging to Avn{12 — 3, J...21 — 
(j + 1)). See also sequences A000085 (j = 1), A001680 (j = 2) and 
A001681 {j = 3) of [S]. Furthermore, we can use to re-derive 
[3], by a limit approach, the enumeration of Avn{n)- Indeed, |Aw„(/i)| 
corresponds to (jl9p ii j = oo. If the parameter j is not bounded, the 
presence of the factor x^ in the second summand of implies that, 
when we plug in a; = 0, the only contribute which remains is given 
by E^^ (0) . This is the n-th coefficient of the exponential generating 
function of the so-called Bell numbers which is in fact equal to 



B{x) 



1 



Bell numbers ((6„)„ in what follows) are listed as sequence ^10001100 
of [9] and, as the reader can easily check in the mentioned reference, 
they give an answer to several enumerative questions. 

Unfortunately it is not so easy to cope with ([T9l) if we want to 
compute the values /j,„for large n. Starting from the fact that n < j 
implies Ij^n = \Avn{lJ,)\ = bn and looking at the decomposition of right- 
left avoiding trees we have used to derive (jl7p . we can define a more 
immediate recursion giving, row by row, the numbers contained in the 
previous table. Indeed we have 
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Proposition 7 For all n > j + 1, the numbers Ij^n can be computed 
recursively as 

^j,n = ( jij,n-i-l, (20) 

i=0 \ * / 

with initial conditions given by lj,i = &i,^j,2 = b2,...,lj.j — bj and 
Ijj+i — t'j+i — 1 (bn is the n-th Bell number). 

Proof. Concerning the initial conditions of tlie recurrence, we just 
observe that among the right-left avoiding trees of size j + 1 there is 
only one tree which must be excluded to compute Ijj+i, that is the 
right chain tree of size j + 1. To obtain ([20|) note that ii n > j + 1, 
then every tree which has to be counted in Ij^n is obtained appending 
a tree counted in Ij^n-i-i as a left sub-tree of a right chain of size i + l, 
where i ranges in [0,j]. □ 

4 Further work: pattern avoidance look- 
ing at sub- permutations 

Using the terminology of Section [5] it is obvious that, for any pattern a, 
each permutation tt satisfies the following equivalence 

TT e Av{a) -^^^^ g^{l) e Av{a). 

This is due to the fact that 57r(l) = tt. This docs in general not hold 
for k > 1. Given fc > 1 and a pattern a, it seems interesting to consider 
permutations tt for which 

TT e Av{a) <^=4> gr,{k) e Av{a). 

In other words we seek tt such that if it contains a then also g-rrik) 
contains a. We denote the set of such permutations by Av{a; fc). Fur- 
thermore note that in terms of probability one has 

P{n e ^t>(cr)|5^(fc) e Av{(7)) ^ P{tt e Av{a]k)) 

which quantifies the presence of the pattern ct in tt depending on its 
presence in the sub-permutation generated by the entry fc. 

As an introductory example we consider the case Av{213; 2). 

The cardinality of Aw„(213; 2) 

We proceed counting those permutations tt g 5„ such that the 
pattern 213 is in tt and g,r(2) € Av{213). Given n of size n, we denote 
by m the lowest entry of tt such that m 7^ 1 and m ^ S7r(2). Observe 
that TO could not exist. In order to have tt ^ A?;„(213;2) only three 
distinct situations are possible. 
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i) The entry 2 is placed on the left of 1 and m exists; 

a) the entry 2 is placed on the right of 1 and g^{m) contains 213; 

in) the entry 2 is placed on the right of 1 and gTvifn) G Av{213). 
Observe that in this case we find 213 in tt if and only if m < M, 
where M is defined as the biggest entry of tt belonging to gn{2). 

Let us denote by i the cardinality of 57r(2), then we have I < i < 
n — 1 and n — i — 1 gives the number of entries in gTr{m). The number 
of permutations corresponding to i) is then 

J2c,{n~^-ly■('lZl)■ (21) 

Similarly, the number of possible instances of ii) is given by 

"^^ / — 9\ 

^c,((n-z-l)!-c„_,_i) r_ J. (22) 

Finally, in case in) we have 

Putting together these three results gives the following 

Proposition 8 The number of permutations of size n wich are not in 
Av„{213;2) is given by 

|5„ \ Av{2U; 2)1 = 2{n - 2)! J[^l^ + ^(n - 2){n - 3)c„_3 

(24) 

+ 2(n - 2)c„_2 - c„ + 2c„_i, 
where c„js f/ie ri-f/i Catalan number. 

It follows that the cardinality of A?;„(213; 2) is given by 

|At;„(213;2)| = |5„\Ai;(213;2)| 
and, for 3 < < 10, the numbers are 

5, 16, 68, 392, 2905, 25508, 251188, 2703440, 

which are not listed in [5]. 
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When n is large we can approximate the sum in parentheses in (|24p 
as a constant 

n — 4 

fc = ^\ ^ 11.75330... . (25) 

An asymptotic approximation of |iS„ \ yli;(213; 2)| is then given by 

2(?i- 2)!fc. 

The probability that a permutation of size n — oo does not belong to 
Av{213;2) is then 

2k , , 

-■ 26 



Asymptotic behaviour of P(7r ^ Aw„(cr;2)) 

Observe that equation (|26|) depends only on the constant k (besides 
n). This constant can be determined with high accuracy from the first 
few, say 50, summands in (f25|) . This suggests that often one does not 
need to know for every n the number of permutations in Aw„(cr) to 
determine the behaviour of Aw„(ct; k)/n\. In fact, if fc = 2 and for n 
large, 

Oh 

P{niAvn{a-2)) = ^, (27) 

where 

is a constant which can be approximated using the first i' values of 
(|^w„(cr)|)„, where i' (the upper summation bound) depends on a. 
The reasoning which leads to (P7|) is the following. As done for the 
case ^w(213;2), let us consider a permutation tt ^ Avn{(J\2). Let i 
be the size of g-n{2) and m the lowest entry of tt with m ^ 1 and 
m ^ s,r(2). There are two basic cases for tt depending on the presence 
of the pattern a in g^{m). If gTrim) ^ Aw((t), the number of possible 
tt's is given by 



\Av,{a)\ |^z;„-,-i(a)| 
(i-1)! ^ (n-i- 1)1 

If gTrini) G Ati(cr) then the number of possible tt's is bounded from 
the top by 




2(n-2)!^ 



\Avi{a)\ \AVn-^-l{(T)\ 
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It follows that 



n-l-\a\ 



2(n-2)! ka 



E 



\Av,Xa)\ \Avn-^-l{a)\ 

X 



^ (i-1)! {n-i-iy. 



< \Sn\Av{<j;2)\ 



(28) 



(29) 



The sums appearing in p8|) and (l29l) go to zero for n large and 
dividing by n\ we obtain the desired result 

\Sn\Avia;2)\ 2fc, 

; ^ — 7^, With n — >■ oo. 

n! 



References 

[1] E. Babson, E. Steingrimsson, Generalized permutation patterns and 
a classification of the Mahonian statistics, Seminaire Lotharingien 
de Combinatoire, v.44, (2000). 

[2] C. Banderier, M. Bousquet-Melou, A. Denise, P. Flajolet, D. 
Gardy, D.Gouyou- Beauchamps, Generating functions for gen- 
erating trees, Discrete Mathematics, v. 246, p. 29-55, (2002). 

[3] A. Claesson, Generalized pattern avoidance, European Journal of 
Combinatorics, v. 22, pp. 961-971. 

[4] A. Claesson, T. Mansour, Counting occurrences of a pattern of type 
(1,2) or (2,1) in permutations. Advances in Applied Mathematics 
29:2 (2002) 293-310. 

[5] P. Flajolet, X. Gourdon, C. Martinez, Patterns in Random Binary 
Search Trees, Random Structures & Algorithms, v.ll, n.3, p. 223- 
245, (1997) 

[6] P. Flajolet, R. Sedgewick, Analytic Combinatorics, Cambridge Uni- 
versity press, (2009). 

[7] P. Flajolet, P. Sipala, J.M. Steyaert, Analytic Variation on the 
Common Subexpression Problem, Lecture Notes in Computer Sci- 
ence, V. 443, p. 220-234, (1990). 

[8] A. Sapounakis, I. Tasoulas and P. Tsikouras, Counting strings in 
Dyck paths, Discrete Mathematics, v.307, p.2909-2924, (2007). 

[9] N. J. A. Sloane, The On-Line Encyclopedia of Integer Sequences, 
available at: http://oeis.org/. 



22 



